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Executive Summary 


The Massachusetts Expanded Learning Time (ELT) initiative was established in 2005 with planning 
grants that allowed a limited number of schools to explore a redesign of their respective schedules and 
add time to their day or year. Participating schools are required to expand learning time by at least 300 
hours per academic year to improve student outcomes in core academic subjects, broaden enrichment 
opportunities, and improve instruction by adding more planning and professional development time for 
teachers. Schools draw upon state resources as well as technical assistance and support from 
Massachusetts 2020 (Mass 2020) and Focus on Results to implement expanded learning time in their 
schools. The first cohort of ten ELT schools (Cohort 1) received implementation grants to begin operating 
their expanded days in the 2006-07 school year; in 2007-08, a second cohort of nine schools (Cohort 2) 
began to implement ELT; and a third cohort of nine schools began in 2008-09, resulting in an initial 
group of 26 1 ELT schools in the Commonwealth. There has not been additional funding for new ELT 
schools since then. In the most recently completed school year, 2010-11, 19 2 schools continued to 
implement the initiative. 

Abt Associates Inc. is completing a multi-year evaluation of ELT that examines both the implementation 
of ELT in the funded schools, and the outcomes for schools, teachers, and students hypothesized to result 
from effective ELT implementation. This report describes current implementation and outcomes for an 
initiative that has been underway for five full academic years. The staggered nature of the ELT initiative 
means that as of the end of the 2010-11 school year, participating schools have completed five, four, and 
three years of implementation (Cohorts 1, 2, and 3, respectively). 


Study Design 

The overall ELT evaluation is guided by three major evaluation questions: 

1. How has ELT been implemented in schools that have received ELT grants? 

2. What are the outcomes of ELT for schools, teachers, and students? 

3. What is the relationship between ELT implementation and outcomes? 

This report addresses all three of the evaluation questions. It focuses considerable attention on how the 
ELT initiative was implemented in the ELT schools during the 2010-11 school year, and also examines 


1 In 2007-08, one Cohort 1 ELT school closed due to restructuring. Also in 2007-08, an existing Cohort 1 ELT 
school merged with a non-ELT school to become a new ELT school; in 2008-09, this new school merged again, 
this time with multiple non-ELT schools, and became a new ELT school. Given the intensity of the 
restructuring this school underwent, ESE changed this school’s cohort designation from Cohort 1 to Cohort 3. 

In 2009-10, two ELT schools, one a Cohort 2 and one a Cohort 3 school, in one district were merged; the 
combined school was designated a Cohort 3 school. During the 2009-10 and 2010-11 school years, six ELT 
schools left the initiative; in three cases the teachers’ union and in two cases the district School Committee 
voted down the school’s continued participation; in one case ESE did not renew the school’s ELT funding due 
to underperformance and because the school’s Level 4 status made it eligible for Federal SIG funding to aid 
with school turnaround. The total number of active ELT schools is as of the time of this report 19, 18 of which 
are included in this evaluation. 
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the effects of the ELT initiative on schools, teachers, and students in the three cohorts of ELT schools for 
three and four years of implementation. 2 3 Finally, the report addresses the third question through a variety 
of descriptive and exploratory analyses of variation in implementation and associated variation in 
outcomes. 

Below, the key findings from the implementation and then outcomes components are summarized. 


Key Findings 

Implementation of Core Components 

In the fifth year of the ELT initiative, all funded schools continued efforts to create a school day that 
incorporated the major elements of ELT: increased core academic time, enrichment opportunities, and 
opportunity for teachers to engage in collaborative planning and professional development. Schools 
varied considerably in their respective efforts to implement the core components. 

Core Academics and Instruction 

• The ELT school day was just under eight hours, on average, in 2010-11. 

• On average, almost five of the nearly eight hours of a typical school day were allocated to core 
academics (English Language Arts (ELA), math, science, and social studies). Twenty more 
minutes per day, on average, were allocated to core academics in 5 th than in 8 th grade. 

• Overall, the plurality of time in an ELT school day was allocated to ELA, followed by math, then 
science and social studies. Specifically, of the five hours allocated to core academics: 

- The amount of time scheduled for English Language Arts (ELA) was 1 hour and 45 minutes, 
on average. 

- Nearly 90 minutes were allocated to math instruction, on average. 

- An average of nearly 1 hour was scheduled for science and 45 minutes to social studies per 
day. 

• Time allocations for core subjects varied somewhat by grade. Specifically, 

- An average of about 45 minutes more each day was allocated to ELA in 5 th than in 8 th grade. 

- Slightly more time (12 minutes, on average) was allocated to math in 5 th than in 8 th grade. 

- About 20 fewer minutes were allocated to science and 15 minutes fewer to social studies in 
5 th than 8 th grade, on average. 

• While there are broad core principles guiding ELT implementation, ELT schools have flexibility 
in how they implement core components. As in past years, schools varied considerably in how 
they allocated time to various instructional activities. 


2 

The report presents findings based on two or three cohorts of schools in the main body; findings based solely on 
one cohort (Cohort 1) are presented, where appropriate, in appendices. 
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School-Wide Academic Focus 

• In the 2010-11 school year, most ELT schools had a school- wide academic focus, according to 
both principals and teachers. Teacher and principal reports of the focus were consistent at 1 1 of 
16 schools. 

• Both teachers and principals reported that the most common focus area was literacy, although 
writing, math, and higher order thinking skills were also common foci. 

• Elementary school teachers reported a literacy-related focus more frequently than middle schools, 
and middle school teachers reported that higher order thinking skills was the focus more 
frequently than elementary schools. 

• Students were most likely to report that the focus was math; however, student reports were rarely 
consistent with those of principals and teachers or internally consistent within a school. 

• At most schools, the focus was posted publicly, often in hallways, teachers’ classrooms, and the 
administrative offices. 

• According to principals, most ELT schools had implemented school-wide instructional practices, 
and the vast majority of teachers reported that their instructional practice was influenced by the 
focus. 

• A substantial majority of teachers reported that they used data specific to the focus area to 
monitor student progress and adjust instructional practices. Most also reported that dedicated 
academic support was influenced by the school-wide focus. 

Enrichment 

• Most ELT schools have implemented separate enrichment classes. Nearly all students participated 
in enrichment classes/instruction, though the amount of time varied. Some schools also embedded 
enrichment activities within core classes. 

• The amount of time a typical student spent in enrichment varied considerably, from daily to 
weekly. Similar to last year, middle school students appeared to spend more time in enrichment 
than elementary students. 

• Approximately half of all ELT teachers reported that they taught at least one enrichment activity. 
Middle school teachers taught enrichment more often than elementary teachers. 

• Regular teachers/staff taught some enrichment activities at most ELT schools, and over half the 
ELT schools relied on partner organizations to provide some enrichment; of those latter schools, 
regular meetings were scheduled for partner staff and teachers to collaborate, an increase over 
reported efforts to integrate partners from the previous year. 

• Most teachers reported that they and their students had some choice about selecting enrichment 
activities. The vast majority of teachers reported that all students had access to enrichment 
activities, and enrichment activities were of high quality. 

Common Planning Time and Professional Development 

• More than half of ELT teachers (65 percent) participated in collaborative planning time weekly or 
more often. 
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• Only a small proportion of teachers (16 percent) reported never having participated in 
collaborative planning. 

• Teachers reported participating in multiple activities during collaborative planning time, 
including analyzing student data, strategizing about instructional practices, and/or reviewing 
student work. The majority of teachers who reported participating in an activity also reported that 
the activity was useful. 

• Teacher perceptions of principal leadership varied across schools. Teachers were more 
consistently positive about principals’ ELT-focused leadership than they were about principals’ 
leadership in general. 

Implementation Index 

The study team developed an index keyed to core principles of effective ELT implementation, as 
articulated by ESE and Mass 2020. The purpose of the index is two-fold: one, to integrate information 
from multiple data sources into one measure that could help to describe variation in schools’ 
implementation efforts, both for individual schools and for the initiative as a whole; and two, to create a 
measure that could potentially be used to explore relationships between level of implementation and 
student achievement. 

The index is based upon interview and survey data from staff and students in both ELT and matched 
comparison schools. Its structure and thresholds reflect contributions from ESE and Mass 2020 as well as 
the study team. The index includes separate scores for eight criteria related to six dimension of 
implementation 4 , and the dimension-specific scores are also combined into an overall index score. Each 
school received a score that ranged from zero (indicating no or very little evidence of implementation on 
a given criterion) to three (indicating consistent evidence of implementation on a given criterion) for each 
of the dimensions, some of which had more than one component. 

Key findings about implementation based on application of the implementation index include: 

• The range of total scores for ELT schools was from 5 to 22, and for comparison schools, the 
range was from 2 to 12 out of a total possible score of 24. 

• The average total score for ELT schools was 11.4 and for comparison schools was 6.9. 

• ELT schools, on average, scored higher than comparison schools on six of the eight criteria. 

• Comparison schools scored notably lower on the enrichment-related criteria. 

• On average, ELT schools’ scores on individual criteria ranged from 1 to 2, and comparison 
schools’ scores ranged from 0 to 1. 

• Many comparison schools also appeared to be implementing at least some of the key components 
that are considered core expectations of the ELT initiative. 


The implementation index dimensions include: school-wide academic focus, core academics in target grades 
(two subcomponents), enrichment activities (two subcomponents), teacher leadership and collaboration (two 
subcomponents), and school leadership, along with ELT-specific stakeholder support. Chapter 3 includes 
additional details about the index. 
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• For two of eight criteria, the average comparison school score was higher than the average ELT 
score. 

Assessment of Outcomes 

Student and Teacher Surveys 

Findings from study -developed surveys are based solely on responses from the 2010-11 school year, 
regardless of individual schools’ implementation year, because the surveys were substantially revised to 
ensure more detailed information from school respondents on time use, teachers’ participation in the 
extended day teaching schedule, and overall time allocations across the schools. Consequently, survey 
responses could not be aggregated with prior survey responses to examine responses as a function of 
implementation year. 

It is also important to note that the study surveyed all teachers across all content areas, and all eligible 5 th 
and 8 th grade students in study schools, and only those schools with response rates above 70 percent for 
both student and teacher surveys are included in findings presented in the report. Teachers from 37 
schools (18 ELT and 19 matched comparison schools) participated in teacher survey administration. The 
response rates across the schools ranged from 64 to 100 percent. Eighteen ELT schools and 17 of the 
matched comparison schools achieved response rates of at least 70 percent 5 . While teacher responses can 
be assumed to be representative of teachers in study schools, student responses cannot, as they represent 
perceptions of students in only one or two grades within schools that serve between three and nine grade 
levels. 

Extant data (e.g., attendance, MCAS scores) 

Findings based on extant data sources are presented in terms of implementation year, as data were 
available for multiple academic years prior to 2010-1 1. For this latter group of outcomes, findings are 
presented in the main body for implementation years one through four (i.e., for schools with two or more 
years of ELT implementation); findings based on five years of ELT implementation (Cohort 1 only) are 
presented in the appendices. 

Analysis of extant data uses a comparative interrupted time series design that leverages pre-ELT data, 
school, and year fixed effects when estimating the effect of ELT. This design is among the strongest 
quasi-experimental designs available, although its analyses are non-experimental. Since schools and their 
students were not randomly assigned to ELT participation, results cannot be attributed solely to ELT. The 
interrupted time series design, use of matched comparison schools and statistical controls, and rigorous 
model specification, taken together, are capable of yielding credible and robust estimates of program 
impacts. This report also presents results from a number of descriptive and exploratory analyses to 
provide context for the comparative analyses; while informative, these findings do not support causal 
conclusions, as they are based upon less robust analyses. 


The two matched comparison schools that were excluded from analyses represent one Cohort 1 and one Cohort 
3 school, and both are elementary schools. 
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Non-Academic Outcomes 

Comparing Time Allocations in ELT and Matched Comparison Schools 

• The length of the ELT school day was significantly longer for 5 th and 8 th grade students than 
would be expected in the absence of ELT. 

• ELT schools allocated significantly more time for ELA, math, and science classes for 5 th and 8 th 
graders than would be expected in the absence of ELT. 

• ELT schools allocated significantly more time for non-core classes and specials for 5 th grade 
students than would be expected in the absence of ELT. 

• ELT schools allocated significantly more time for enrichment activities for 5 th and 8 th grade 
students than would be expected in the absence of ELT. 

• ELT schools allocated significantly more time for transitions, recess, snack, lunch, and 
homeroom for 5 th and 8 th grade students than would be expected in the absence of ELT. 

• A statistically smaller proportion of students in ELT schools reported that they attend an 
academic club than would be reported in the absence of ELT. 

• Significantly fewer students in ELT schools attended an after-school program than would be the 
expected in the absence of ELT. 

Teacher Outcomes 

• A significantly higher proportion of teachers in ELT schools reported that the length of the day 
allows them to accomplish their teaching goals and cover the amount of instructional material 
their students need to learn than would be expected in the absence of ELT. 

• A significantly higher proportion of teachers in ELT schools reported that they are satisfied with 
the amount of time available for instruction in ELA, math, and science than would be expected in 
the absence of ELT. 

• A significantly higher proportion of teachers in ELT schools reported that they are satisfied with 
the amount of time available for academic support, enrichment activities and for students to 
pursue topics of interest than would be expected in the absence of ELT. 

• Significantly more teachers in ELT schools reported that they are satisfied with the amount of 
time available for collaborative planning and that the length of the day allows for coordination of 
instruction than would be expected in the absence of ELT. Conversely, significantly fewer 
teachers in ELT schools than the counterfactual reported that the amount of collaborative 
planning time is a problem area. 

Student Outcomes 

• A significantly higher proportion of teachers in ELT schools reported that teachers and students 
spend sufficient instructional time together than would be expected in the absence of ELT. 

• Significantly more teachers in ELT schools reported that teacher and staff fatigue, as well as 
student fatigue, were problem areas than would be expected without ELT. Likewise, a 
significantly higher proportion of students in ELT schools reported that they were tired in school. 
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• Significantly fewer students in ELT schools reported that: they look forward to going to school; 
like being in school; that all of their classes are important to them; and that they like the length of 
their school day, than would be expected without ELT. 

• A significantly smaller proportion of teachers in ELT schools reported that student academic 
performance and homework completion rates were problem areas. 

• Students in ELT schools had statistically significantly higher suspension rates than would be 
expected in the absence of ELT ; however, while statistically significant, the differences were 
extremely small in magnitude, and therefore are unlikely to have educational or practical 
significance. 

Student Achievement Outcomes 

• In the first and second years of implementation, ELT schools served a statistically significantly 
greater proportion of minority students than estimated in the absence of ELT, although the 
estimated magnitude of the differences (3.7 and 4.0 percentage points, respectively) is unlikely to 
be practically meaningful, and there were no effects of ELT on schools’ minority student 
population in the third or fourth year. 

• In the third year of implementation, ELT schools had a statistically significantly smaller 
proportion of highly qualified core academic teachers compared to the estimated proportion in the 
absence of ELT (2.9 percentage points). 

• In the fourth year of implementation, ELT schools had a statistically significantly lower number 
of FTE teachers (4.7 fewer), and statistically significantly higher student-teacher ratio (almost 
two more students per teacher) than estimated in the absence of ELT. 

• Across all years of implementation, there were no significant differences in average student 
mobility rates between ELT and matched comparison schools. 

• Descriptive analyses restricted to ELT schools indicated variation in student performance levels 
among schools both before implementation began and in the most recent school year (2010-11), 
and indicated no consistent patterns of results. Descriptive analyses indicated that some schools 
have substantially increased the percentage of students that reached proficient or advanced 
performance levels, while others have experienced little change or decreased percentage of 
students at these same levels. 

• On average, there were no statistically significant effects of ELT after one, two, three, or four 
years of implementation on MCAS student achievement test outcomes for 3 ld , 4 th , or 7 th grade 
ELA, 4 th , 6 th , or 8 th grade math, or 8 th grade science. 

• There was a statistically significant positive effect of ELT after four years of implementation on 
the MCAS 5 th grade science test. 

• Exploratory analyses using data from non-ELT schools in ELT districts, and non-ELT schools 
statewide, rather than the study’s matched comparison schools, were generally consistent with the 
primary analysis, including the significant finding for 5 th grade science noted above. In addition, 
there was a statistically significant negative effect of ELT on 3 ld grade reading after two years of 
implementation, and there were statistically significant positive effects of ELT on 6 th grade math 
and 8 th grade science after four years of implementation in both the district-level and state-level 
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analyses. The state-level analysis also found a statistically significant positive effect of ELT on 
7 th grade ELA after one year of implementation and on 6 th grade math after three years of 
implementation. 

• Exploratory descriptive analysis linking the level of implementation in ELT schools and student 
achievement outcomes indicated no clear patterns or meaningful relationships. 

• Exploratory analysis investigating the difference of the effect of ELT in higher- versus lower- 
implementing schools indicates minimal heterogeneity in the effect by the level of ELT 
implementation. However, the effect of ELT on 8 th grade math in higher-implementing schools is 
estimated to be statistically significantly greater than the effect of ELT in low- implementing 
schools after three and four years of implementation. 


Discussion 

Across findings from interviews, surveys, and achievement data, the following themes seem clear: 

• There is strong evidence that the ELT schools have implemented many core ELT elements, both 
in terms of additional time available for instruction, academic support, and enrichment and 
supports for teachers’ use of that time. 

• There continues to be substantial variation across ELT schools’ level and approach to 
implementation (as measured by interviews, surveys, and an index). 

• Measuring different aspects of time use is challenging: collecting information on a prototypical 
student in a given grade level may or may not reveal how students are supported by the ELT 
initiative and definitions of various activities/time uses are not consistent across schools. 

• There are some, but not many, differences — even descriptively — between ELT and comparison 
schools on survey and achievement outcomes. 

• The school reform landscape is dynamic and more schools (outside of this ELT initiative) appear 
to be expanding the amount of time in their school year, as well as implementing reforms 
consistent with the core ELT components with each successive year. 

• This study was able to assess the quantity and allocation of time, but did not measure the quality 
of instruction, enrichment, and other activities made possible by the additional time, and clearly, 
the quality of such activities is also important. 


Future Steps for the ELT Initiative 

The ELT initiative has been underway for several years, and can now be considered a fairly mature 
intervention. Over that time period, as the schools’ implementation efforts have matured, the contexts 
within which the schools operate have continued to change. Some of that change reflects increasingly 
explicit guidance from the Massachusetts Department of Elementary and Secondary Education, some 
reflects the increasingly targeted nature of technical assistance and support from Mass 2020 and Focus on 
Results, and some reflects the heightened visibility of ELT more broadly, through other federal and state 
initiatives such as School Improvement Grants and Race to the Top funding. While the federal, state, and 
local contexts have changed, and the implementation of the initiative in the ELT study schools has also 
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continued to evolve, the improved academic achievement outcomes for students have not materialized as 
expected across ELT schools as a whole. 

This multi-year evaluation has described schools’ ongoing implementation efforts in four prior interim 
reports. ELT schools clearly have made progress on implementing many of the core elements of ELT and, 
as measured by the study’s implementation index, score higher than the matched comparison schools on 
average. Yet the patterns of implementation differ as much across the ELT schools as between the ELT 
and comparison schools, highlighting the variation in ELT across the initiative. The fact that such 
variation exists in the initiative’s fifth year illustrates both the complexity inherent in large-scale efforts to 
transform low-performing schools and the variation that inevitably results from flexible interventions that 
can be shared to fit individual schools’ needs. It may also reflect the different motivation of schools to 
participate in ELT from the outset, as some schools opted in voluntarily while others were strongly 
encouraged to apply. 

Variation in ongoing implementation is clearly a continuing theme for the Massachusetts ELT initiative. 
Prior years’ impact analyses have found little evidence of effects on students’ academic achievement, and 
the results from the fifth year indicate that students’ academic achievement outcomes, on average, have 
largely remained unaffected. Descriptive and exploratory analyses provide limited suggestive evidence 
that student growth in ELT schools is greater than growth in non-ELT counterparts, yet such results are 
not generally statistically significant. Despite the demonstrable progress ELT schools made to implement 
core components of ELT, those implementation efforts have not yet consistently translated the additional 
time into the content, strategies, or support that in turn yield improved overall student performance. 
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